In many applications of machine learning and pattern recognition, it is often easy to design an algorithm to perform a particular task given aligned input data. For example, if one is trying to perform character recognition and the characters always appear in exactly the same fashion, it is quite simple for any method to correctly determine which character is being seen. However, the input data often has undergone some sort of transformation before being seen by the system, for instance, if the characters are written by two different people or if the surface on which they are written is viewed from a different angle. These variances, or transformations, cause degradations in the performance of systems that rely on aligned data to perform their task. Similarly, when dealing with faces in tasks such as face detection, facial landmarking, or face recognition, the face can have wildly differing appearances based on the viewpoint of the camera. When dealing with simple distortions, such as a projective transformation, it is a fairly simple process to normalize the image. However, a projective transformation normalizes planar surfaces but not any 3D object in the scene.
Reconstructing the 3D structure of any general object cannot be done from a single two-dimensional (2D) image due to ambiguities in the projective geometry of the camera. However, under certain assumptions, this ambiguity can be resolved. In the case of 3D craniofacial reconstruction, the interest is only in reconstructing the 3D structure of a person's head, which drastically reduces the possible 3D objects that could be seen when looking at the 2D image. Some methods have already shown how a 3D reconstruction of the face can be generated from a single 2D image. The 3D Generic Elastic Models (3D-GEM) technique by Heo and Savvides (J. Heo and M. Savvides, 3-d generic elastic models for fast and texture preserving 2d novel pose syntheses, IEEE Transactions on Information Forensics and Security, 7(2):563-576, April 2012) was shown to be able to generate very accurate 3D reconstructions of the face from a single, frontal 2D image. These renderings were even used to improve face recognition results later by Prabhu et al. (U. Prabhu, J. Heo, and M. Savvides, Unconstrained pose-invariant face recognition using 3d generic elastic models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10):1952-1961, October 2011) showing how using the 3D information of the face can be very useful in other tasks, such as face recognition. This has been built upon by many others in recent works.